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Abstract 

We introduce the notion of order of magnitude reversibility (OM-reversibility) in Markov 
chains that are parametrized by a positive parameter e. OM-reversibihty is a weaker con- 
dition than reversibihty, and requires only the knowledge of order of magnitude of the 
transition probabilities. For an irreducible, OM-reversible Markov chain on a finite state 
space, we prove that the stationary distribution satisfies order of magnitude detailed balance 
(analog of detailed balance in reversible Markov chains). The result characterizes the states 
with positive probability in the limit of the stationary distribution as e — > 0, which finds an 
important application in the case of singularly perturbed Markov chains that are reducible 
for e = 0. We show that OM-reversibility occurs naturally in macroscopic systems, involving 
many interacting particles. Clustering is a common phenomenon in biological systems, in 
which particles or molecules aggregate at one location. We give a simple condition on the 
transition probabilities in an interacting particle Markov chain that characterizes clustering. 
We show that such clustering processes are OM-reversible, and we find explicitly the order 
of magnitude of the stationary distribution. Further, we show that the single pole states, 
in which all particles are at a single vertex, are the only states with positive probability in 
the limit of the stationary distribution as the rate of diffusion goes to zero. 

Keywords: reversibility, detailed balance, Markov chains, clustering, pole formation, 
interacting particle systems, singularly perturbed Markov chains. 



1 Introduction 

This paper has two objectives. The first is to introduce the notion of e-order of magnitude 
reversibility (OM-reversibility) in a family of Markov chains X'', parametrized by e > 0. The 
condition of OM-reversibility is weaker than reversibility, and requires only the knowledge of 



order of magnitude of the transition probabilities. The main result in this article (Theorem 3.6) 
gives the order of magnitude of the probabilities in the stationary distribution of an irreducible, 
OM-reversible Markov chain on a finite state space. The order of magnitude of the unique 
stationary distribution tt^ on for e > is sufficient to characterize the set of states with 
positive probability in the limit of the stationary distribution as e — ?■ 0. The second objective of 
this paper is to characterize clustering processes. Clustering processes are interacting particle 



systems, in which the particles have a tendency to one location (we will refer to 

the phenomenon of clustering to a single location as pole formation). For an interacting particle 
system to be a clustering process, we only require that the probability for a particle to move 
to an adjacent, unoccupied vertex is an order of magnitude smaller than the probability to 
move to an adjacent, occupied vertex (see Definition 4.3). We prove that clustering processes 
are OM-reversible and we explicitly give the order of magnitude of the probabilities in the 
stationary distribution for all particle configurations (Theorem 4.12). 

At microscopic or atomic scales, time-reversibility or simply reversibility is frequently a fun- 
damental property of physical systems. Mathematically, reversibility is formulated as detailed 
balance or as the Kolmogorov cycle condition, see Theorem |2.1[ If a stochastic process is re- 
versible, then a stationary distribution exists and further the detailed balance condition can be 
solved to give the stationary distribution explicitly. 

At macroscopic scales, such as the ones that occur in cell biology involving multiple inter- 
ating particles, reversibility is less likely to be encountered. On the other hand, there may be 
examples of systems that are 'approximately reversible' and we might expect that if a stationary 
distribution exists, then it satisfies a corresponding version of 'approximate detailed balance'. 
In the same spirit, we introduce the notion of order of magnitude reversibility (OM-reversibility) 
in Markov chains (see Definition 3.2), where the order of magnitude is with respect to a positive 
parameter e. We prove that the unique function (up to an additive constant) that satisfies the 
OM-reversibility condition in a finite, irreducible Markov chain is the order of magnitude of 
the stationary distribution. Furthermore, we show that for an irreducible Markov chain on a 
finite state space, OM-reversibility is equivalent to a condition that we call order of magnitude 
Kolmogorov cycle condition or OM- cycle condition. 

We illustrate how OM-reversibility can arise in macroscopic systems through an example 
of a class of interacting particle systems that we will refer to as clustering processes. The 
phenomenon of clustering occurs frequently in biological systems. To give an instance from 
cell biology, cells that are initially spatially symmetric, can spontaneously lose symmetry and 
evolve into an asymmetric state with molecules clustered together at one spot. Bud formation 
in a yeast cell is initiated when Cdc42 molecules one location on the surface of the 

cell [U 131 [m |22] . Besides yeast, hippocampal axons [23]; canine kidney cells [9], and human 
chemotaxing neutrophils show clustering of specific molecules resulting in cellular polarity. 
Other examples and models from the biological literature can be found in [5l 171 IIOI 1141 127 t I28j . 
A somewhat different example of pole formation is the firing frequency of neurons in a network 
aggregating to one value, making the population of neurons fire coherently. 

We show that, for the clustering processes, the size of the support, defined to be the number 
of occupied vertices in the network, satisfies OM-detailed balance. Thus the size of the support is 
the order of magnitude of the stationary distribution, up to an additive constant. Of particular 
interest is the identification of the states that have a positive probability in the stationary 
distribution in the limit e — )• 0. Consider, for instance, a Markov chain X'' which is irreducible 
for e > but reducible for e = 0. A natural question is, "Which one of the multiple stationary 
distributions on is lim^_^o+ '^'^ where vr*^ is the unique stationary distribution on X*^?" Since 
single pole states (states with exactly one occupied vertex) minimize the size of the support. 
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only single pole states have positive probability in the stationary distribution in the limit 
e — 7- 0. As another example, we look at clustering processes with carrying capacity, which is a 
generalization of the clustering processes. 

Biologically detailed models of clustering include a model involving a set of coupled partial 
differential equations |11J as a model for yeast cell pole formation. Altschuler et al. [I] propose a 
model for spontaneous emergence of cell polarity using only the mechanism of positive feedback; 
detailed mathematical analysis of the model was carried out by Gupta [12J. Markov chains 
parametrized by e have been studied in the past using perturbation techniques. Schweitzer [2^ 
studied the perturbation expansion of the stationary distribution when the Markov chain is 
irreducible for all values of e > 0. Lasserre [17] generalized Schweitzer's formula to the case of 
singularly perturbed Markov chains. Latouche and Louchard [18] studied a case of singularly 
perturbed Markov chains, where the Markov chain is irreducible for e > but is reducible for 
e = and decomposes into disjoint aggregates of states. Avrachenkov and Haviv [2] studied 
the coefficients of the first terms in the Laurent series of the first return time in the case of 
singularly perturbed Markov chains. Hassin and Haviv [ISj provided a combinatorial algorithm 
for computing the order of magnitude in e of the mean passage time and the first return time 
in a set of Markov chains parametrized by some e > 0. Interacting particle systems on a graph 
called zero range interaction processes have been studied in [201 ESI ES] where a particle jumps 
from a vertex x to an adjacent vertex y with a probability that depends on the occupancy of 
X. Clustering processes are cousins of zero range interaction processes, because for clustering 
processes the probability for a particle to jump from x to y depends on the occupancy of x 
along with the occupancy of all its neighbors including y. 

This article is organized as follows. Section [2] provides an overview of basic results on re- 
versibility in Markov chains and gives two equivalent characterizations of reversibility (Theorem 



2.1). Section ^ defines order of magnitude reversibility and states the main theorem (Theo- 



rem 



3.6) that the stationary distribution on an OM-reversible Markov chain satisfies the order 



of magnitude detailed balance condition. Section [4] studies the application to clustering and 



pole formation; Definition |4.3| provides the definition and Theorem 4.12| gives the stationary 



distribution of clustering processes. Section [5] defines a generalized version of the clustering 



processes, clustering processes with carrying capacity (Definition 5.1), and Theorem |5.4| gives 
the order of magnitude of the stationary distribution of such processes. Section [6] provides 
numerical simulations and observations about the behavior of the Markov chain for small rate 
of diffusion. 

Notation 1.1. Throughout this paper, fort £ Z, X{t) represents a Markov chain, ^1 represents 
the state space of X and P the transition matrix of X. We will say that the triple {X{t), il, P), 
or simply {X,Q,P), is a Markov chain, vr denotes a stationary distribution on X. When we 
consider a family of Markov chains, parametrized by e >0, we represent a member of the family 
as {X'^,Q,P^). When a stationary distribution exists, it will be denoted as tt'^. 
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2 An overview of reversibility 



We begin with a brief discussion of the concept of time-reversibility, often known simply as 
reversibility. Outside of the stochastic process setting, time-reversibiUty plays an important 
role in many fundamental laws of physics at the microscopic scale. In chemical reaction network 
theory, microscopic reversibility gives rise to the important idea of detailed balance [HI ITOl |2T| [30] . 
Casimir extended the idea of detailed balance to electric networks [4J . Within the field of Markov 
chains, there are a number of applications of reversibility, many of which are studied in |16j . In 
the next theorem, we state two equivalent ways of characterizing reversibility, detailed balance 
and the Kolmogorov cycle condition. 

Theorem 2.1. (Time reversibility |^ I For an irreducible Markov chain (X,Q,P), the 
detailed balance condition is equivalent to the Kolmogorov cycle condition. In other words, the 
following are equivalent. 

1. There exists a function on the state space vr : — )• M>o satisfying the detailed balance 
condition 

7r(x)P(x, y) = 7r{y)P{y, x) for all x,y eQ. (1) 

2. For every finite sequence of .states {xi, X2, ■ . . , Xn-i, Xn = xi) C 0,, the following Kol- 
mogorov cycle condition holds 

n—l n—1 

llP{xi,x,+i) = 11 Pi 

1=1 1=1 

If either of these conditions is satisfied, we say that {X, $7, P) is reversible. 

Defining the notion of probability flux from state x to state y as 7r{x)P{x,y), the condition 
for TT to be a stationary distribution is simply that the flux into state x i.e. 7r(y)P(y, x) and 
the flux out of state x i.e. 7r(x)P(x, y) = 7r(x) are equal. The detailed balance condition is 
a stronger condition that requires that for any two states x and y, the flux from state x to state 
y is equal to the reverse flux from y to x. The advantage of reversibility is that it guarantees 
existence of a stationary distribution in a Markov chain and allows explicitly identifying it. 

In this paper we relax the condition of reversibility, and deflne the weaker notion of order of 
magnitude reversibility. Every reversible Markov chain is order of magnitude reversible, but the 



converse is not true. As we show in Theorem 3.6, order of magnitude reversibility is sufficient 



to give orders of magnitudes of the probabilities of states in the stationary distribution. 

3 Order of magnitude reversibility 

The main result in this article is that if tt is a stationary distribution on an OM-reversible 



Markov chain (Definition 3.2), then the order of magnitude of the stationary distribution vr 
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satisfies a simple additive identity ([s]) that is analogous to detailed balance in reversible Markov 



chains (Theorem 3.6), and we refer to the identity as order of magnitude detailed balance or 
simply as OM-detailed balance. We also show in this section that for an irreducible Markov chain 
over a finite state space, OM-detailed balance is equivalent to order of magnitude Kolmogorov 
cycle condition (Theorem 3 



3.1 Preliminaries 

A function / : M+ ^ M+ is called e(e'=) for some E Z if there exist Mi, M2, e E M+ such that 
for all e with < e < e, Mie'^ < /(e) < M2e'^ holds. Define the order of magnitude function 
If, : R+^+ ^ Z by ipeif) = kif and only if / is G(e^). 

When it is clear from the context, we will drop the subscript e from the order of magnitude 
function and simply write ip for ip^. 

Lemma 3.1. The order of magnitude function satisfies the following properties 
1- '/'(/i + /2) = min{(^(/i), 99(72)}. 

2. ^{fl-f2)=V{fl)+^{f2). 

Proof. Both properties follow easily from the definition of 93. □ 

Definition 3.2. We say that the Markov chain {X,0,,P) is order of magnitude reversible if 
there exists an integer valued function v : 0, ^ Z such that for all n, f E 17 with P{u, v) > 0, v 
satisfies the following order of magnitude detailed balance condition 

u{u) + ^{P{u, v)) = v{v) + ip{P{v, u)). (3) 

Remark 3.3. It is implicit in the definition that P{u,v) > if and only if P{v,u) > 0. 

Defining (j) := cp o P, the reversibility condition ([s]) can be rewritten as 

u{u) + (p{u,v) = u{v) + (t){v,u) (4) 

We will refer to order of magnitude reversibility as OM-reversibility or alternatively as OM- 
detailed balance from here on. 

Theorem 3.4. If an irreducible Markov chain is reversible, then it is OM-reversible. 

Proof. For x, y E such that P{x, y) > 0, the detailed balance condition ([T]) holds. Applying 
99, it is evident that v = 99(vr) satisfies ([s]). □ 

Lemma 3.5. For an irreducible, OM-reversible Markov chain {X,Vt,P), if there exist ui and 
V2 satisfying then vi — V2 = c, a constant. 

Proof. Let z^i, 1/2 : 17 — ?• Z be two functions satisfying OM-reversibility ^ and let u, w E 17. By 
irreducibility, there exists (ui, M2; • • • , Un-i) C 17 such that nr=o^ P{''^i: ^j+i) > where uq := u 
and Un := v. Then for all i E {0, 1, . . . , n — 1} and j E {1, 2}, Vj{ui) + (p{P{ui, Uj+i)) = Vj{uiJ^i) + 
if{P{ui+i,Ui)). So that i'i{ui)-i'i{ui+i) = Jy2{ui)-i^2{ui+i)- Summing over i E {0, 1, . . . ,n-l}, 
we get i^i(ii) — i'2{u) = i'i{v) — 2^2 (f). So that i^i — 1/2 = c, a constant. □ 



5 



3.2 Stationary distribution on an OM-reversible Markov chain 

We now state our main theorem about the order of magnitude of the stationary distribution on 
an OM-reversible Markov chain. 

Theorem 3.6. (Stationary distribution satisfies OM-detailed balance) If ir is a stationary dis- 
tribution on a finite, irreducible, OM-reversible Markov chain {X, P) then for all x,y £ 0, 
such that P{x,y) > 0, tt satisfies OM-detailed balance i.e. 

ipi-Kix)) + (t){x, y) = ip{TT{y)) (l){y, x) (5) 

where (p = ip o P. 

Proof. Let : 17 — t- Z be an integer-valued function satisfying ([s]) such that min^gf^ = 0. 
Clearly, ly is unique by Lemma |3.5[ Let o" : 17 — )• Z where 

a{x) := if{-K{x)) — z^(x). 

In order to prove the theorem, we will show that a = 0. Let 5 C 17, and S'^ := 17 \ 5. Let 
dS := {x G S\P{x,y) > for some y G S'^}. The stationary distribution on any Markov chain 
satisfies the following identity (see for instance [16]) 

xedS yedS" xGdS yiadS" 

Noting that vr > for a finite, irreducible Markov chain, compose with ip and use ip{'jr(x)) = 
a{x) + iy{x) to get 

min min (cr(x) -|- -|- (p{x, y)) = min min {a{y) + v{y) + (p{y, x)). 
x^dS y&dS'^ x&dS y&dS'^ 

Using OM-reversibility, we write ^{y) -\- (j){y, x) as -|- (j){x, y) on the right hand side, which 
gives 

min min {a{x) + v{x) + (f){x,y)) = min min (cr(y) -|- v{x) + (f){x,y)). (6) 
x&dS y&dS'^ x&dS y&dS'^ 

Suppose by way of contradiction that a ^ c where c G Z is a constant. Let m = min^jgn 
and let S = {x £ VL\a{x) = m}. S is non-empty by construction and since cr is non-constant, 
is also non-empty. 

m + min min {v{x) + 4>{x, y)) = min min {cr{x) -\- i/{x) -\- (j){x, y)) 
x£dS yGdS'^ x^dS y&dS'^ 

= min min (cr(y) + z^(x) -|- (j){x,y)) (by M 
x&dS y&dS'^ 

> min cr{y) -\- min min (z^(x) -|- (j){x,y)) 
y£dS'^ xGdS yGdS'^ 

> m + min min ivix) + (bix, v)) 

xedSyedS'^ 

which implies the contradiction, m > m. So the assumption that a is non-constant must be 
false; implying that a = c. Finally, = minx^n ip{TT(x)) = c + min^.^^ i'{x) = c + = c, and so 
a = 0, which completes the proof. 

□ 
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3.3 Order of magnitude Kolmogorov cycle condition 

Analogous to reversible Markov chains, we define order of magnitude Kolmogorov cycle condition 
or OM-cycle condition as an alternative way to characterize OM-reversible Markov chains. 
The OM-cycle condition only requires knowledge of the orders of magnitude of the transition 
probabilities in each cycle in the graph corresponding to the Markov chain. The OM-cycle 
condition gives a direct way to check OM-reversibility, since it does not require constructing a 
1/ as in the case of OM-detailed balance. 

Definition 3.7. If for every finite sequence of states {xi,X2, ■ • • , Xn-i, Xn = xi) C 17 such that 
Y\a=i P{xi,Xij^i) > 0, the following condition holds 

n—l n—1 

= (7) 

i=l i=l 

then we say that (X, 0, P) satisfies order of magnitude Kolmogorov cycle condition or OM-cycle 
condition. 

Theorem 3.8. Let (X, Q, P) be an irreducible Markov chain. X is OM-reversible if and only 
if X satisfies the OM-cycle condition. 

Proof. Suppose first that X is OM-reversible, so that there exists a : O — ?• Z which satisfies 
v{u) + (t){u, v) = i'{v) + (j){v, u) for all u, w G J7 such that P{u, v) > 0. Let {xi,X2, ■ ■ . , Xn-i,Xn = 
xi) C be such that nr=i^ P{xi,Xi-\.i) > 0. Then OM-reversibility implies that 11?=/ Pixi+i,Xi) > 
and 

n— 1 n—l n—l 

(l){Xi,Xi+i) - ^ (j){Xi+l,Xi) = ^ {(piXi, Xi+l) - (j){Xi+l,Xi)) 
i=l i=l 1=1 

n-1 

= - iy{Xi)] = v{Xn) - vixi) = 0. 

i=l 

Conversely, suppose that for every finite sequence of states (xi, X2, • . . , Xn-i-, Xn = xi) d Q. 
such that \YiZi P{xi,Xi-\.i) > 0, the OM-cycle condition ([T]) holds. Let x,y £ Q. Since X is 
irreducible, there exists at least one sequence of states 7 = (2/1,2/2, ■ • • ,2/n-i) C such that 
nr=o^ Piyi,yi+i) > where yo := y and y„ := x. Define 

n-1 

Kx,y) = fJ'{x,y;'j) ■■= ^ [(t>{yi,yi+i) - 4>{yi+i,yi)] . 

i=0 

To see that fi is independent of the sequence 7, let y2n '■= y and suppose there is another 
sequence 

7 := {yi^y2^---^y'n-2^y'n-i) ■= {y2n-i,y2n-2, ■ ■ ■ ,yn+2,yn+i) c n 

such that U7=o Piy'vy'i+i) > 0- Then 
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n— 1 n— 1 

= [4'{yi,yi+l) - HVi+liVi)] - X] [Hy2n-i~l,y2n-i-2) - 4>{y2n-i~2,y2n-i-l)] 



i=0 i=0 

n—l n—1 



[4>iyii yi+l) + 4>iy2n-i-2, y2n-i-l)] - ^ [</'(y2n-i-l, y2n-i-2) + (/"(yj+l, yj)] 
1=0 i=0 
2n-l 2?i-l 

X] 4>{yi,yi+i) - ^ 4>iyi+i,yi) = o. 



i=0 i=0 

where we used the OM-cycle condition ([T]) in the last step. 

It is easy to check that for x,y, z £ fi{x, y) + z) = /i(x, z) and ^(y, x) = — y). 
Fix a state xq ^Vt and for a; G il, define 

i/(x) := /i(x,Xo) - min//(y,xo). (8) 

To see that v is well-defined, let Xq / xq. So 
/i(x,Xo) - min//(y,Xo) 



= /i(x, xo) + /x(xo, xg) - min(//(y, xq) + ^(xq, Xq)) 

y 

= h{x,xq) - min/i(y,xo). 

y 

Let u, u G such that v) > 0. Then 

v{u) - u{v) = h{u,xq) - fJ.{v,xo) 
= /x(n, xo) + ti{xo, v) = fi{u, v) = (p{v, u) - (j){u, v). 

which shows that X is OM-reversible. □ 



Remark 3.9. Theorem \3.8\ shows that for a finite, irreducible Markov chain, we can either take 
order of magnitude Kolmogorov cycle condition ([T]) or order of magnitude detailed balance Q 
as the definition of OM-reversibility. The irreducibility condition is without loss of generality, 
because for reducible Markov chains we can focus attention on just the communicating, closed 
subset of states. 



3.4 Characterization of the graph associated with the transition matrix of 
an OM-reversible Markov chain 

Now that we have established our main result on order of magnitude of the stationary distri- 
bution, we make an observation about the structure of the graph associated with the transition 



8 



matrix P"" of an irreducible, OM-reversible Markov chain. We recall that such a graph is ob- 
tained by taking the vertex set to be the set of states O and the weighted edges to be the 
elements of the transition matrix, with the weight proportional to the probability of transition. 
In particular, a transition with probability zero corresponds to an edge with weight zero, or 
equivalently to no edge at all. In order to establish the connection between OM-reversibility 
and the structure of the graph, we partition the state space as follows. 

Definition 3.10. Let Qn = {x € Q\ip{7:(x)) = n} 

Clearly U^g^i is a partition of Q. Suppose we draw the graph associated to so that n is 
the y-coordinate (height) of the elements of r2„. For m,k > 0, let x £ $7^ and y G ^m+k be such 
that P{x,y) > 0. Then ^{x,y) — (j){y,x) = ip{T:{y)) — ip{'ir{x)) = k >0. In particular if A; > 0, 
then the probability of a transition from x to y is at least one order of magnitude smaller than 
the probability of a transition from y to x. In other words, downward transitions are more likely 
than upward transitions. This is the geometric characterization of an OM-reversible Markov 



chain. We refer to Figure 4.5 for a particular example of an OM-reversible Markov chain, where 
such a property is evident. 

Since is the set of states with positive probability in the limit e — )• 0, when considering 
applications of OM-reversibility it is important to characterize fio, the set of states on which v 
(a function that satisfies OM-detailed balance) attains a minimum. 



4 Application: Clustering and pole formation 

The phenomenon of aggregation of particles to a single location, known as pole formation, is 
the motivation for defining and studying OM-reversibility. It is of great interest to determine 
the fundamental principles of pole formation because it occurs in a wide variety of biological 
systems. We refer to a process that results in pole formation as a clustering process. To give 
one specific instance of a cellular clustering process, yeast cells can sometimes develop a bud 
on the surface, which initiates growth of the yeast at the budding site. The bud formation 
itself is initiated when molecules of Cdc42, initially scattered across the surface of the cell or 
within the cell, start aggregating at one location jllj . This and many other such phenomena 
in cell biology [H |3l EJ d El [22l [2ll [28l [29] led us to study models of interacting particle system 
Markov chains where pole formation occurs. We found that the key property that many such 
interacting particle systems shared was that of OM-reversibility. 

Thus clustering processes are a natural choice as the first example of OM-reversibility. Con- 
versely, clustering processes provide evidence of the usefulness of the notion of OM-reversibility. 
While reversibility may be a rare property in macroscopic systems such as interacting parti- 
cle systems, we argue that OM-reversibility is much more common. This is because OM- 
reversibility requires only a mild condition on the order of magnitude of the transition proba- 
bilities. 

In this section, we provide sufficient conditions on the transition matrix of a Markov chain 
for pole formation. We study processes where at each time step a single particle jumps from a 
vertex x to an adjacent vertex y, the probability of this jump depends on the occupancy of the 
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vertex x and the occupancies of all the neighboring vertices of x including that of y. Processes 
where the probability of a jump depends only on the vertex x, called zero range interaction 
processes^ have been studied in [20l EH [26]. A process where the probability of transition 
depends on the occupancy of the vertex x and on the occupancy of the vertex y, but not on 
the occupancy of the other neighbors of x, was studied in [15j . 



We define clustering processes in sections 4.1, 4.2, and 4.3 and in section 4.6 we show 



the existence of an integer-valued u satisfying OM-detailed balance, thus establishing OM- 
reversibility of clustering processes. 

4.1 Network structure 

Recall that a network N = {V, E) is a finite, undirected, connected graph with vertex set V 
and edge set E. If there is an edge connecting the vertices Vi and Vj we will say that Vi and Vj 
are adjacent and write Vi ~ Vj or occasionally i ^ j, meanwhile the edge itself will be denoted 
by the unordered pair {vi,Vj}. In the rest of this article, we consider an underlying network 
N = {V,E) with \V\ := m. A vertex can be occupied by multiple particles, n being the total 
number of particles in the network. We label the vertices of the network vi, . . . ,Vm- 

4.2 State space 

The state space of the Markov chain X'^{t) consists of all possible configurations of the n 
particles among the m vertices {vi, . . . ,Vm) of the network. More precisely, a configuration or 
a state is an ordered collection of non- negative integers (xi, X2, . . . , Xm) such that < Xj < n 
and YliLi Xi = n. Denote the set of all states hy := ^n,m- 

Example 4.1. Consider the case of m = 4 vertices and n = 3 particles. ^^3^4 consists of 4 
permutations of (1, 1, 1, 0), 12 permutations of (2, 1, 0, 0) and 4 permutations of (3, 0, 0, 0). The 
total number of states is 1 1^3,4! = 20. 

Definition 4.2. For distinct integers i,j £ {1, . . . ,m}, if x,y £ fl are such that yj = Xj + 1, 
Vi = Xi — 1, yk = Xfc for k £ {1, . . . , m} \ {i,j}, we will write y = x*'-^ . To make later definitions 
easier to write, we will allow x*'* = x. Note that if y = x*'-^ then x = y^'^ . 

We refer to a stochastic process involving multiple particles on a network N as an interacting 
particle system. 

4.3 Definition of clustering process 

Definition 4.3. We define a clustering process to he an interacting particle Markov chain 
(X^, fi, P'^) where consists of all configurations of n particles on an arbitrary network N and 
where for all x such that Xi > 1, and for Vi ~ Vj or for i = j, satisfies 

= { J f^^. (9) 
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4.4 Models of clustering process 



4.4.1 Clustering tendency 

We define a function / that we will refer to as clustering tendency. 
Definition 4.4. Let f : R>o x Z>o — > M>o be such that 

I £1 \ \ f 1 if X = 

We give some examples of the clustering tendency /. 

Example 4.5. In the following examples the diffusion strength e is nondimensional in order to 
make the discussion about small e meaningful. Some examples of the clustering tendency f are 

1- /o(e,p) = I 1' ^{f^pZ^Q (step function). 
P 

2. fi(e,p) = — he (linear). 

n 

3. /2(e,p) = ^— ^ +e (quadratic). 

At each time step, we pick a particle uniformly at random, and move it either to one of the 
adjoining vertices or return it to the original vertex Vi. Each of these jump events occurs with 
a probability that is proportional to the clustering tendency /(e, xj) where vj is the destination 
vertex. In Model 1, the probability of a jump depends on the origin vertex Vi and the destination 
vertex vj. In Model 2, the probability of a jump depends on the origin vertex Vi and all its 
neighbors Vk ~ Vi including the destination vertex Vj. For a fixed e, we let := /(e, •). 

We will define the probability of transition from the state x to the state x^'^ where Vj ~ Vi. 
P{x,x) is then determined from J2v ^i^^v) — ^■ 



4.4.2 Model 1 - Interaction between the origin and the destination site 

The following example is a slightly modified version of the process studied by Joshi et al. [15j. 
The probability of a jump depends on both the origin vertex and the destination vertex. On a 
d-regular network, define for j ^ i, 

^^"^'"^ ' d n f,{x,) + f,{x.) ^'"^ 
Moreover, for x,y G we define P^{x,y) := lime_j.o -P'^(x, y). 
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4.4.3 Model 2 - Interaction between the origin site and its neighbors including 
the destination site 

We introduce an example of a clustering process where the probability of transition depends on 
the origin vertex, and all its neighbors including the destination vertex. On any network N, if 
Vj ~ Vi, then 



For x,y £ Q we define P {x, y) := lim^^Q P'^(x, y) 



Theorem 4.6. The process defined by the transition matrix (11) in Model 1 and in Model 2 is 
a clustering process. 

Proof. We prove the statement for Model 2, since the proof for Model 1 is quite similar. If 

Xi > 1, then < feiMxi) + Y^kr^i fe{Xk)) < (Pe{fe{Xi)) = 0. So (feifeiXi) + Y^k^i f^i^k)) = 

and 

^.(F-(..."))=^.(/.(.,))={ J . 

This proves the theorem. □ 

4.5 Structure of the graph associated with the transition matrix of a clus- 
tering process 

In this section we describe the structure of the graph associated with the transition matrix 
of the Markov chain for the clustering processes ([9]). We consider the instance where the 
underlying network consists of m = 4 vertices arranged in a circle. In other words, each ver- 
tex has precisely 2 neighbors. Moreover, there are n = 5 particles. Even for this relatively 
simple example, the state space is quite large, \^\ = 56. We will use the dihedral symme- 
try of the underlying network to define a new, but closely related Markov chain. The state 
space of the new Markov chain X consists of equivalence classes of states. Two states are 
considered in the same equivalence class if they have the same neighborhood structure. In 
other words if one state is in the orbit of the other state under the action of the dihedral 
group Dg, then the two states are equivalent. For instance, one of the equivalence classes is 
{(1,1, 3,0), (0,1, 1,3), (3, 0,1,1), (1,3, 0,1), (0,3, 1,1), (3, 1,1,0), (1,1,0, 3), (1,0, 3,1)}. We will 
select an arbitrary representative of the equivalence class to denote the entire equivalence class 
when referring to a state in Vt. The details on defining the transition probabilities of the new 
Markov chain with this symmetry can be found in [15]. 



The new Markov chain X*" has = 10 states shown in Figure 4.5 We represent the 
Markov chain as a graph whose vertices are states and a directed edge from state x € to state 
y G O represents i-*(x, y) > 0. The edge is represented in red (dark) if (j){x, y) = and in green 
(light) if (j){x.,y) = 1. We have drawn the graph associated with the Markov chain so that the 
vertical coordinate of the state x is the order of magnitude of the probability of the state in 
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the stationary distribution ip{TT{x)). Because P is OM-reversible, all downward arrows are red 
(dark) (probability of transition is @{€^)) and all upward arrows are green (light) (probability 
of transition is 0(e^)). 

For e = 0, the absorbing states of are the 1-pole state, (0, 0, 0, 5), and the 2-pole states, 
(0, 1, 0, 4) and (0, 2, 0, 3), i.e. absorbing states of are the ones for which all outgoing edges 
are green (light) or G(e^) 



We show in Theorem 



4.12 



that only the single pole state (0, 0, 0, 5) 
has positive probability in the stationary distribution vr*^ as e ^ 0^. 



(1, 1, 1, 2) 








' 






(0, 1, 3, 1) _ 


" (0, 1, 2, 2) 


* (0, 1, 1, 3) 


(0, 2, 1, 2) 




l-l 




fl \\ ">.. 





n 




(0, 0, 1,4) 




(0, 0, 2, 3) 




(0, 1, 0, 4) 




(0, 0, 0, 5) 






/ 

(0, 2, 0, 3) 



L. 



Figure 1: Markov chain for m = 4 bins and n = 5 balls. The red (dark) arrows indicate that 
<j){x,y) = while the green (light) arrows indicate that (j){x,y) = 1. The vertical coordinate of 
the state x € is determined by lp{tt{x)). Of the three absorbing states (0, 0, 0, 5), (0, 1, 0, 4) 
and (0, 2, 0, 3) of the reducible Markov chain X^ , only the single pole state (0, 0, 0, 5) has positive 
probability in the stationary distribution vr"^ of the irreducible Markov chain X'' as e — t- 0"*". The 
figure was drawn using Mathematica. 



4.6 Pole formation in the clustering processes 

In this section, we show that a clustering process is OM-reversible, even though not reversible 
in general. We first establish that the clustering process is irreducible and aperiodic for positive 
e. 

Theorem 4.7. For e > 0, a clustering process X'^{t) is irreducible and aperiodic. 
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Proof. Irreducibility follows because the underlying graph is connected and for e > each 
neighboring vertex is accessible with positive probability. Aperiodicity is almost immediate 
since this only requires that there is an x S such that P{x, x) > 0, which is in fact true for 
all X e 0. □ 



Remark 4.8. We should emphasize that the Theorem J^.l is true only for e > 0. For e = 0, a 
clustering process is reducible and has multiple absorbing states. In fact for e = 0, any state for 
which the occupied vertices are isolated is an absorbing state. In other words, if x £ Q is such 
that if Xi > 0,Xj > implies that Vi oo vj then x is an absorbing state. 

Since for e > 0, a clustering process X*^ is irreducible and aperiodic, there exists a unique 
stationary distribution n = ir^ with Tr''{x) > for all x £ Q. We now give a simple characteri- 
zation of the probability of a given configuration in the stationary distribution in terms of the 
number of particles at each vertex. 

Definition 4.9. The support of the state x £ ^1 is defined to be s{x) := {vi € V\xi > 1}. The 
number of occupied vertices or the size of the support of the state x £ Q is the cardinality of 
the set s{x), denoted by \s{x)\. 

Lemma 4.10. For a clustering process, let x,y £ ft be such that y) > 0. Then (f>{x, y) = 
if and only if s{y) C s{x). 

Proof. If <j){x,y) = then either y = x or y = x^'^ with Xj > 0. In either case, s{y) C s{x). On 
the other hand, if (f){x,y) = 1 then y = x^'^ with xj = 0, so that s{y) <f. s{x). □ 

Corollary 4.11. Let x,y £ Q with P^{x,y) > 0. 

1. If 4>{x, y) = and (j){y, x) = then s{y) = s{x). 

2. If (l){x, y) = and (f){y, x) = 1 then s{y) ^ s{x) and \s{y)\ = \s{x)\ — 1. 

3. If <j){x, y) = 1 and 4>{y, x) = 1 then \s{x)\ = \s{y)\. 

Proof. If = (j){y,x) = then s{y) C s{x) and s{x) C s{y), which shows case 1. If 

(j){x,y) = and 4>{y,x) = 1 then s{y) C s{x) but s{x) ^ so that s{y) ^ s{x). Moreover, 
\s{y)\ < \s{x)\ and since only one particle is moved \s{x)\ < \s{y)\ + 1; combining the two 
inequalities we have \s{x)\ = \s{y)\ + 1 which proves case 2. Finally, if 4)[x,y) = (p{y,x) = 1 
then \s{y)\ > |s(x)| and \s{x)\ > \s{y)\ which shows case 3. □ 

Theorem 4.12. A clustering process X'^(t) is OM-reversible and for x £ ^l, 

(P{tt{x)) = \s{x)\ - 1. 

Proof. We need to find a : — )• Z that satisfies Q. If such a u exists then, (/^(vr) — u = c. For 
X £ Q, let ^{x) = \s{x)\ — 1. From Corollary 4.11| 



z/(x) - u{y) = \s{x)\ - \s{y)\ 



, if (l)iy,x) - (l)ix,y) = 

1 , if (t>{y,x) - (t>{x,y) = 1 
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In either case, v[x)—i'{y) = (f){y^x) — (j){x,y). Furthermore, min^jgfj = m\nx,^Q\s{x)\—l = 0, 
when there is at least one particle in the network. This proves the theorem. □ 

Corollary 4.13. VLq = {z ^ : \s{z)\ = 1}. 



Corollary [4l3l det ermines VLq for the clustering processes to be the set of single pole states, 



i.e. states for which all the particles are accumulated at a single vertex. It is natural to ask 
the question, "which other processes besides the clustering processes have the set of single 
pole states as the limiting stationary distribution Oq?" We define a generalization of clustering 
processes which provides the answer. 

Definition 4.14. An interacting particle system on a network, for which at most one 
particle jumps to an adjacent vertex at each time step, is called a generalized clustering process 
if the transition matrix satisfies the following 

1. For all X, y such that P{x, y) > 0, (p{P{x, y)) G {0, 1}. 

2. If\s{x)\ = \s{y)\, then ^{P{x,y)) = ip{P{y,x)). 

3. If\s{x)\ = \s{y)\ - I, thenip{P{x,y)) = 1 and ip{P{y,x)) = 0. 



A clustering process is a generalized clustering process as is clear from Corollary 4.11 We 



give two examples of generalized clustering processes that are not clustering processes. 

Example 4.15. 1. Suppose the transition matrix P satisfies the condition that for all states 
x,x^'^ such that P{x,x^'^) > 0, (p{P{x, x"^'^)) = if and only if Xi = 1. In other words, the 
only event that does not have probability 0(e) is the event where a particle leaves an empty 
vertex in its wake. This process is a generalized clustering process but not a clustering 
process. 

2. A slight variant of a clustering process is one where the transition probabilities obey Q 



in Definition 4-3 with the exception that when Xj = 1 and xj = 0, then ip{P{x,x^'^)) = 0. 
In other words, a particle moves to an unoccupied vertex with probability Q{e) unless it 
leaves an empty vertex in its wake, in which case the probability of transition is 0(1)- 

An explicit example of such a process is obtained as follows. On a d-regular network, 
define for j ^ i, 

d n f^{xj) + fe{xi - 1) 



An interpretation of the extra '—1' is that once a particle is picked, the probability of return 
to the vertex of origin depends only on the number of particles remaining. This process 
is defined and analyzed in / i 5/ . where it is shown to be reversible. Obviously, the process 
is OM-reversible. Due to the extra '—1 ', the process is not a clustering process but it is a 
generalized clustering process because when Xi = 1 and xj = 0, then ip{P{x, x^'^) = 0. 
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Theorem 4.16. A generalized clustering process is OM-reversihle and the stationary distri- 
bution has order of magnitude \s[x)\ — 1. Conversely, let X'^ be an irreducible, OM-reversible 
process on a finite state space such that for all x,y & ^l, (p{x, y) G {0, 1} and such that the 
stationary distribution has order of magnitude \s{x)\ — 1. Then is a generalized clustering 
process. 

Proof. For a generalized clustering process \s{y)\ — \s{x)\ = (p{P{x,y)) — ip{P{y,x)), which 
proves the first part of the claim. 

Conversely, OM-reversibility of X^ implies that if \s{y)\ — \s{x)\ = 1 then (p{P{x,y)) — 



^{P{y,x)) = 1. Since ifiP) e {0,1}, ip{P{x,y)) = 1 and ^{P{y,x)) = 0. If \s{y)\ - \s{x)\ = 



then ip{P{x, y)) — ip{P{y, x)) = 0. This shows that X'^ is a generalized clustering process. □ 

5 Application: Clustering with a carrying capacity 

In this section, we consider an extension of the clustering processes, namely clustering processes 
with a 'soft' carrying capacity. We assume the same network structure, a connected, undirected 
graph, with n particles initially distributed among the vertices. We assume that the particles 
have a tendency to cluster except when there are too few particles at a vertex or when there 
are too many particles at a vertex. For the vertex Vi, if the occupancy is under Lj, or over the 
carrying capacity Ki, a particle can arrive at Vi only with probability that is B(e). 

Definition 5.1. Let < Lj < Kj for all j. We define the clustering process with carrying 
capacity to be an interacting particle system (X^, Q, P^) where the state space O consists of all 
configurations of n particles on a connected network and where for Vi ~ Vj or for i = j, P^ 
satisfies 



The ordered pair {Lj,Kj) will be referred to as the carrying capacity of the vertex vj. 
5.1 Examples of clustering with ceirrying capacity 

We consider the simplest case where each vertex has the carrying capacity {L,K), in other 
words, Lj = L and Kj = K for all Vj G V. 

Definition 5.2. Define the clustering tendency / : M>o x Z>o — > M>o to be 



As before we think of e as diffusion. We present some examples of the clustering tendency 



^,{P'{x,x''^)) 



J 1 , if Xj < Lj or Xj > Kj 
\ , if Lj < Xj < Kj 



(13) 




Example 5.3. Once again e is assumed to be nondimensional. 
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1. For < L < K < n, h{e,p) 



, ifpe{L,K) 



2. ForO<L<K<n, h{e,p) = | A'-l + ^ ' f ^ (^'^) . 

[ e , otherwise 

3. ForO<L<K<n, /2(e,p) = max | o} + e. 

Let the transition matrix be as defined in Model 1 (equation (10)) or as in Model 2 



(equation ( |Tl| ) ) . Then it is easy to check that is a clustering process with carrying capacity. 

5.2 Stationary distribution in clustering with carrying capacity 

Theorem 5.4. If tt is the stationary distribution for a clustering process with carrying capacity 
{L, K) for all vertices v € V, then 

m 

if{-K{x)) = [min(a;;, L) + max(x; — K + 1,0)] — c 
1=1 



where c = min < > [min(x/, L) + max(x; — K + 1,0)] > . 



1=1 



Proof. Let fi{x) = Ylh=i [min(x/, L) + max(3;^ — K + 1,0)]. We will show that /i satisfies OM- 
reversibility. 

fi{x) - n{x''^) 

m m 

= [min(xi, L) + max(xi — K + 1,0)] — |^min(xj''', L) + max(x;'-' — K + 1,0) 



1=1 



1=1 



[min(xj, L) + max(xj — K + 1,0)] + [min(xj, L) + max(xj — K + 1,0)\ 



min(x*''', L) + max(j;*''' — K + 1,0) 

Ti + Tj 



mm 



xy , L) + max(x*'^ -K + 1,0) 



where 



Ti := [min(a;j, L) + max(xj — K+1,0)] — [min(xj — 1, L) + max(2;j — K, 0)] 

Tj := [mm{xj, L) + max(xj — K+1,0)] — [min(xj + 1,L) + max(xj — K + 2,0)] . 



Ti 



and 



1 , \i Xi < L OT Xi > K 
, if L < Xi < K 



— 1 , il Xj < L — 1 or Xj > K — 1 
, ilL-Kxj < K -1 

Moreover, the following relations are true for the transition probabilities. 
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1. li Xi < L or Xi > K and Xj < L — 1 or xj > K — 1, then (j){x^'^ ,x) = 4>{x, x^'^) = 1. 

2. U Xi < L or Xi > K and L-l<Xj<K-l, then 4>{x^'^ ,x) - <j){x, x"^'^) = 1-0 = 1. 

3. If L < Xi < K and Xj<L—lorxj>K— 1, then (j){x^'^ ,x) — (j){x, x*'-' ) = — 1 = —1. 

4. li L < Xi < K and L-l<Xj<K-l, then (j){x'-'^,x) = x*'-') = 0. 

In all the cases, we have fi{x) — ^(x*'-') = (j){x'^'^ , x) — (j){x, x*'-'), which shows that n satisfies 
OM-reversibility. So that ip{-ir{x)) = — miuy ij,{y) . □ 



Theorem 5.5. Let fi{x) be as defined in the proof of Theorem 5.J^. Let z G Oq. Then the 
following statements hold: 

L Ifm> Lf J then = L[^\+ min {L,n-K J ) . Further, 

(a) If n — K [jj-J < L then z ^ if and only if z is a configuration for which [-^J 
vertices contain at least K particles. 

(h) If n — K \_^\ > L then z £ i^o if and only if z is a configuration for which 
vertices contain exactly K particles and one vertex contains the remaining n — K 
particles. 



2. If m < |_-^J then fi{z) = n — m{K — L) and z is a configuration such that all vertices 
contain at least K particles. 

Proof. All the cases are obtained by maximizing the number of particles that are between the 
lower threshold L and the upper threshold K. □ 

A clustering process is a special case of a clustering process with a carrying capacity for 
K > n and L = 1. We recover Theorem |4.12| as a corollary of Theorem 5.4 



Corollary 5.6. For a clustering process, ip{Tr{x)) = \s{x)\ — 1. 

Proof. For K > n, max{xi — K + 1,0) = and so with L = 1, defined in the proof of 
Theorem 15.41 is 

m 

fj,{x) = min(x/, 1) = 
1=1 

Furthermore, for K > n, [-^J = and so by Theorem 



5.5 



min fj,{y) = L 
y 



n 

min(l, n) = 1, 



+ min { L,n — K 



n 



as long as there is at least one particle in the network. So ip{'K{x)) = fi{x) — miuy ij,{y) = 
|s(x)|-l. □ 
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6 Numerical studies of a clustering process 



Theorem 4.12 suggests that for the clustering processes with small values of the rate of diffusion, 
particles should cluster to a single vertex in the network. We simulated a particular instance of 
a clustering process, for which the underlying network is a torus of dimensions (20, 12). Initially, 
half the vertices are occupied with one particle and the other half are empty; empty vertices 
alternate with the occupied ones. The transition matrix is given by equation (11) in Model 2 
with the clustering tendency fe{p) = (p/n)'^ + £• We used a diffusion of e = 0.0005 and ran the 
simulation for a total of 1, 000, 000 time steps. In Figure [2] we show successive snapshots of one 
simulation taken at times 1000, 40000, 80000 and 300000. As time progresses, we see particles 
accumulating at one site. 





Figure 2: We consider a torus of dimensions (20, 12), initially vertices containing one particle 



alternate with unoccupied vertices. The transition matrix is given by equations ( 11 ) in Model 2 
with the clustering tendency fe{p) = (p/n)'^ + e, where n = 120 and e = 0.0005. The snapshots 
are taken at times 1000, 40000, 80000, and 300000. 

Define the peak ratio p{t) to be maxj |^^^^^|, the number of particles in the vertex with 
maximum occupancy divided by the total number of particles. In Figure |3j we plot the support 



19 



size and the peak ratio p{t). As clustering to a single pole takes place, the support 

size decreases to a value close to 1, while the peak ratio increases to a value close to 1. For 
the simulation corresponding to Figure [Sj we calculated the average value of the support size 
over the last 500,000 time steps and found that this quantity was 1.2219. We calculated 
the average value of the peak ratio p{t) over the last 500, 000 time steps and found that this 
quantity was 0.9981. 




Figure 3: The support size |s(^)| approaches a value close to 1 as the particles cluster at one 
site, as depicted in the figure on the left. In the figure on the right we plot the peak ratio p{t), 
which approaches a value close to 1. All the graphs in this section are plotted using Mat lab. 
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